Markov processes on Riesz spaces

Markov processes on vermiculated spaces

A general technique is given for constructing new Markov processes from existing ones. The new process and its state space are both projective limits of sequences built by an iterative scheme. The space at each stage in the scheme is obtained by taking disjoint copies of the space at the previous stage and quotienting to identify certain distinguished points. Away from the distinguished points,...

ON THE INFINITE ORDER MARKOV PROCESSES

The notion of infinite order Markov process is introduced and the Markov property of the flow of information is established.

Solving Markov Decision Processes in Metric spaces

We present an approximation scheme for solving Markov Decision Processes (MDPs) in which the states are embedded in a metric space. Our algorithm has a time bound of Õ(n log logn ), where n is the number of states and the approximation factor. This bound is independent of the numerical size of the input and the discount factor and is hence strongly polynomial. We present the result for determin...

Concentration inequalities on product spaces with applications to Markov processes

Bisimulation on Markov Processes over Arbitrary Measurable Spaces

We introduce a notion of bisimulation on labelled Markov Processes over generic measurable spaces in terms of arbitrary binary relations. Our notion of bisimulation is proven to coincide with the coalgebraic definition of Aczel and Mendler in terms of the Giry functor, which associates with a measurable space its collection of (sub)probability measures. This coalgebraic formulation allows one t...